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ABSTRACT 

The Paris Faculty of Science is developing programs 
in computer-assisted instruction (CAI). Their first goal is to 
develop "guest Ion nail' a* (instructional sequences) administered by 
teletype machines which Chech on a student’s Knowledge in an area and 
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modified the taste hardware system and developed appropriate computer 
languages and software systems, which are described here. . >th this 
equipment, the teaching teai has developed questionnaires for CAI in 
aathematics, physics, biology , and English (linguistics) for college 
undergraduates. Tne scope of these programs ard steps in pjoiucing 
then are discussed. Revision of these programs is currently taking 
place. (JR) 



ED050594 









ORGANISATION FOR ECONOMIC 
CO-OPERATION AND DEVELOPMENT 



Centre for Educational Research 
and Innovation 



Paris, 51 March, 1971 

Or. fr. 



CERl/CT/70. 60 « ■. DtPAFTMfNl OF HEALTH. EDUCATION 

» WIUA« 

OFFlCC OF IDUCATIC’N 
THIS DOCUMENT HAS SEEN P1PAOD4JCED 
EXACTLY AS RECEIVE!/ FROM THE PERSON CM 
ORGANIZATION ORIGINATING IT FO NTS OF 
VIEW OR OP1N«ONR STATED DO NOT NECES 
SAFlLr RtKOTSET'OfFK'ALOfFICF OMDU 
CATKIN POSITION OR POl.CY 



CONFERENCE ON THE USE 



OF COMPUT E RS IN HIGHER EDUCATION 

held in Paris from 19th-21st March, 1970 
(Joint Project CERI XI) 



Computer assljted instruction 



by Professor LE CORRE and Professor R. JACOUD and 
their research teem from the 
’♦Computer for Students 1 ’ Laboratory 
of the Paris Faculty of Science (France) 



79*524 




2 



TABLE OF CONTENTS 



INTRODUCTION 

I. PROBLEMS THAT AROSE AND SOLUTIONS FOUND 




1 * PROBLEMS THAT AROSE , ? 

A, Teaching problems 7 

B, Technical problems . • . * g 

2 * SOLUTIONS POUND 

A, The research team 10 

B, Solutions in teaching \2 

C, Technical solutions 13 



II. MATERIALS USED 



1 . THE USER'S POINT OP VIEW 19 

A. Author Language - (A.L. ) . % . „ 

1* Question block , , . 19 

2 . Answer block * » , 20 

* B. Associated description language (A.D.L. ) .... 21 

1. Symbols used 22 

2 . Extending the language 23 

(i) Further remarks ... * * 23 

* (ii) Indicators vor flip-flops) 23 

(iii) Co'inteis 24 

(Iv) Branching by means of indicator or counter 

tests 24 




- 1 - 



3 , 



3. A concrete example of use . . . . 25 

4 . Criticism and outlook .......... 23 

C. How answers are analysed 23 

1 . Without analysis . . ............ 28 

2 . Verba.', analysis 29 

{i) .Purpose 29 

(ii) PillirLg out a typical answer 29 

(iii) Options 30 

(iv) Examples ............... 30 

3* Analysis of formulas ........... 31 

D. BCM - Information system during questioning 30 

1 . Description 32 

2. Definition keys 33 

3 . Capacity - 33 

4. Student operating control ... 33 

5 . Author operating control . 34 

6 . Single-character messages . . ...... 34 

7 . Inventory of recorded definitions ... 34 

8 . Recording requests for information 34 

E. Programmes in use . 35 

1# Orgadiag programme 35 

2 . Punching the question cards 35 

3 . Listing . . 35 

4 . Recording and issuing unexpected answers 36 

5. Handling and processing student data . 36 

2. HOW MATERIALS PIT INTO THE SOFTWARE SYSTEM 

A, Basic system 36 

1 , Modifications made in the B.O.S. 3 ystem ........ 37 

(i) Local modifications ........... 37 

(ii) Writing a programme to handle messages for the 1052 38 

3. Application programme .......... 38 



O 



- ? 



II. APPLICATIONS 



1. HOW THE PROGRAMME ./AS APPLIED TO PHYSICS 41 

A. Perfecting the questionnaires 41 

B, Writing the questionnaires 42 

1 . General goals 42 

2 . Method used 43 

3. Sone i-'les for design 44 



2. USING MIN IT RAN AND SYMPLE LANGUAGES, DISCUSSION CLASS 



WORK IN MATHEMATICS • - 

A. Minitran and how it is used • 46 

B. Synplo and how it is used 48 

C. Mathematics discussion section ... 50 

3 . BIOLOGY 

A, The group of biologists 61 

B. The subject matter 61 

C. The main features of the questionnaires 52 

D, Results and student response 54 

£. Teaching staff's opinion .... 57 

1 . Reservations expressed • 37 

2 . Enthusiasm 37 



4. AN EXPERIMENT IN USING A COMPUTER TO TEACH ENGLISH 
LINGUISTICS TO UNDERGRADUATES ( RULES; OPERATIONS; 
METALANGUAGE) 

CHARLES V ENGLISH INSTITUTE - STUDENT COMPUTER USE 



AT THE HALLE- AUX -VI NS CAMPUS 

A. Problems involved •}& 

B. The programme? * 60 

C. What was accomplished • 61 



IV. summary AND CONCLUSIONS 



A. Make-up of the preaent research team • 63 

B # Defining the problem • 63 



ERIC , 

d / cr 



C« What the " computer’ 1 team has accomplished 64 

D. What the teaching team has accomplished . . 65 

B. Nature of research work on teaching . . * . • 67 

1* Nature of the teacher- student dialogue . 67 

2 m Nature of teaching strategy 67 

3. Nature of a learning model ........ 63 

F. Conclusion 68 

1. The cost-efficiency ratio . . . ^8 

2. Th© choice of subject matter 69 

3. The choice of hardware 70 

4 . Improvements in software 70 

5. Research on teaching 71 

V. PUBLICATIONS 73 



,6 



INTRODUCTION 



The following document, prepared by the "Computer for Students” 
team of the Paris Faculty of Science Directors (Professor Y. lie Corre 
and R. Jacoud) from a more extensive report, describes the various 
aspects of the c omputer-assisted instruction experiment conducted in 
this Laboratory. The experiment forms part of the Joint Project on 
"The Use of Computers in Higher Education" undertaken by the OECD 
Centre for Educational Research and Innovation (CERl). Other studies 
will be published by CERI, each describing the various experiments now 
proceeding under this Joint Project, in which the following laboratories 
and university centres, as well as the team mentioned above, are 
concerned; 

University of Cambridge, Department of Applied 
Mathematics and Theoretical Physics, Director; 
Prof* O.K, Batchelor. 

University of Louvain, General Physics Laboratory, 
Director: Prof. A. Jones. 

University of leiden, Department of Education, 
Directors Prof. 1. de Klerk. 

University of Osaka, Department of Education, 
Directors Prof. S. Tanaka. 

The objectives of this Joint Project are numerous and have been 
described in detail in several documents already issued by CERI. It 
seems, however, advisable to recall the main points: 

1* Preparation of recommendations for a policy on computer utilisation 
In education 



United Kingdom : 

Belgium ; 

Netherlands ; 

Japan s 



Experiments are too often abortive or linger on without producing 
interesting results. The reasons for these relative failures vary 
considerably according to the case, tut one thing is nevertheless 
certain: the novelty of this teaching "tool" on the one hand, its 
obvious and often fascinating possibilities on the other, and the need 
to include it in more comprehensive thi diking about teaching and learning 
processes, should lead to its utilisation being considered on the basis 
of development-oriented educational research. Responsibility for such 
research will ultimately devolve on the educationalists and the national 
authorities. That is why it is important to help them to define their 
options and make their decisions - this is indeed one of CERI's major 
roles - by providing them with the best possible information. 
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2. Co nsideration of a number of questions specific to the use of 

CL .aputero as teaching instruments 

For the teacher as well as for the student, the dialogue with a 
computer is in fact the simulation of an extremely complex teaching 
situation. Computer-assisted instruction constitutes a reduction of 
this. Numerous questions arise - among them the following: 

(i) How should the educational con t ent be organised ? Are there 
one or more "possible approaches", and in the latcer case, 
are some of them optimal? 

(ii) How should the course "input" to t he computer be arranged ? 

In other words, a teacher cannot be expected to be a highly 
skilled computer scientist but, conversely, the "language" 
which he is to use to communicate with the computer should 
be sufficiently rich and flexible to enable him i o express 
the subtleties and various aspects of the teaching process. 

(iii) How should the student's behaviour be taken into account ? 

For instance, how shculd his replies to the computer's 
questions be handled and analysed? To stick to "multiple- 
choice" systems soon becomes an over-facile solution. 

(iv) Is there an ideal "c pnf iguration" f or computer-assisted 
instruction ? What are the advantages and disadvantages of 
the various types of terminals? 

(v) H ow should the computer's role as a teaching instrument be 
" assesse d" .from loth an educational and an "economic" view- 
point? In other words, is computer-assisted instruction 
profitable? 

(vi ) Ho w should the computer be integrated into the educational 
p rocess ? The problems involved are not only technical or 
pedagogic but also sociological. 

The following document does not claim to give a definitive answer 
to all these questions but - and this is to its credit - it describes 
in detail an experiment which, although still unfinished, nevertheless 
forms an instiu^ent of reference. 
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I . PROBLEMS THAT AROSE ANjJ SOLUTIONS FOUND 



1, PROBLEMS THAT AROSE 



A, Teaching problems 

The massive influx ^of students for the Certificat d' Etudes 
Superieures d 'Electricite (undergraduate electricity course) led us, 
by 1965, to wonder about modernising our teaching methods. At that 
time, we were responsible for providing high quality instruction for 
2,000 students, using a faculty of 17 \3 full profersors, Id junior 
faculty) . 

The teaching programme was divided into large lecture courses, 
small discussion groups (of >0 to 55 students), snd oral questioning 
sessions. Although the arrangement was considered satisfactory as far 
as the small discussion groups were concerned, the oral testing system 
by no cleans met with expectations, that is to say, it did not offer 
vn o ~ly conversations between student and teacher, during which the 
acquired knowledge and understanding a re explored, action then taken 
to remedy weaknesses, and if need be the student sent to review the 
course work. 

Of all the possible levels to w.e remedial action at lecture 
courses, discussion groups or testing sessions, we chose the testing 
session. This choice was dictated by the following observations. 

V/hen a student gives a report, any error or mi sunder standiiig 
generally shows up well after the error in reasoning has been committed. 
The student should therefore be corrected before the error appears. The 
job of finding the exact point at which the student began to go wrong 
can only be accomplished by a staff which is highly qualified and 
thoroughly experienced in teacbi*ig, in short, staff which is already 
assigned elsewhere {to cover the discussion groups) and is irreplaceable. 
In these circums tances, the oral questioning sessions were handled by 
graduate students or by students from special elite schools (Grandes 
Ecoles). Though they were knowledgeable enough, they were woefully 
lacking in teaching experience, so that the testing sessions, at that 
time, turned into mere monologues, and they had no purpose other than 
to provide grades (whose value was left in doubt). 

Hence we thought that an automated system would allow us to solve 
the problem of cheoking on the students' knowledge* We stated the 
problem in this manner: 



- Imagine a communications system which would aJlow the student to 
know accurately, regularly and without wasting time, how his 
knowledge stood, and which would draw his attention particularly 
to basic concepts, fundamental definitions and important theorems. 

Starting with this definition, two paths are open> Either one con- 
fines oneself to a mere diagnosis (which means a simple examination 
system), or else one tries to step in at the very moment a mistake is 
made in order to correct, re-explain, and provide extra Information 
(which means what we have called self-examination) . 

In view of the possibilities of the tool (a computer) which we were 
hoping to get, we quite naturally agreed upon the latter path. This 
choice implies answers to three commanding requires ents : 

- allowing the broadest possible answers, ,. s 

- permitting the fullest possible branching, x 

- making it possible to perform computations during the examination. 

Once this framework was outlined, it was necessary to think about 
using such a system at two levels: 

- The immediate practical level: 

- impact on the contents of the lecture coulee 

- impact on the students themselves 

- The basic research level: 

- to begin with, by developing the study of errors, correlations 
between errors,, etc... 

Making use of this system meant developing a means of recording 
statistical data such as 

- the headway made by the student 

- the number of correct answers 

- the number of wrong answers 

- the time taken to answer 

- unexpected answers 

- answers students give to questions about their reactions (how 
sure they are of an answer, for example). 



B» Technical problems 

The technical problems that arose as we carried out the experiment 
we had planned could be tackled frou; various angles, but it was 
important for us to solve then in a way that did not hinder the projects 
development. Every new teaching problem hr.d to be expressed and solved 
within t ha Chosen technical environment. 
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Furthermore, in that we were among the first groups^ to attempt 
to raise this kind of problem, it was not possible for us to make x^se 
of prior experiments and results obtained under similar conditions. 

Although we had set modest goals, they seemed too ambitious for 
us to be ab^e to \.se electromechanical systems such as " teaching 
machines' 1 . A certain kind of programming (linear or divided) must be 
decided on once end for all before such a system can be adopted, so 
that this equipment creates immense difficulties if it has adjusted to 
meet radical changes in procedure, and it cannot be used to analyse 
free-lance answers. 

Apart from the practical problems involved in setting up and 
financing a system based on a computer, it offeis features which best 
match the problems which come up in teaching: 

- the possibility of storing a large volume of data on rapid, 
random access surfaces (diskpacks and drums) (2) 

- using the computer on a time-sharing basis makes it possible 
for a large number of users to be connected to one central 
facility, 

- storing information in real time, so that it can be used during 
sessions (access to information about all the students) or 
afterwards (using standard computer facilities). 

- the possibility of using a wide variety o' terminals, such as 
typewriters, viewing screens, sound readers on magnetic tapes# 

These features, along with the fact that a computer offers much 
greater theoretical flexibility than a "teaching machine' 1 does, led us 
to make the choice we did. 

Once the choice was made, however, all the technical problems 
connected to implementing plans and using the computer remained, namely, 
writing the necessary software programmes# 

Some basic software-assembler, compiler, supervisor - can be 
supplied by the constructor, but this component is rarely adapted to 
any particular application, especially if it is not a common one# Thus, 
it is necessary *co go about defining and working out a system and 
programmes specially designed for the experiment at hand. Thus, ve had 
to chocse between two alternative solutions: 

- modify existing software, with all the "risks' 1 that using a 
"rebuilt" system carries - the most economic but the least 
trustworthy option, 

- write a new, suitable system. 



(1) Outside of experiments done in the USA, but having a philosophic 
basis different from ours (teaching done by computers, teaching 
directed by computers or else various simulations by computers;* 

(2) We decided against using tapes because their access is not rapid 
or simultaneous. 
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Finally, for the problem of choosing a programming language, there 
are also two possible approaches: 

- a machine language (assembler), 

- one of the developed languages (Fortran, Algol, Lisp). 

The choice depends to a great extent on the system which the 
constructor supplies and on the central storage capacity available. 

If only limited performance is required, a developed language 
can be used. In this way the time needed to write and debug the 
programmes is cut down. 

In the light of these considerations, the job of programming can 
be classified in three categories: 

- the operating programme for the terminals - it depends a lot 
on the type of terminal chosen and on whether or not there is 
a time-snaring monitor supplied by the constructor. 

- the programme for processing and recording data collected - 
statistical breakdowns, learning models, etc... 

- the programme for organising the dialogue with the student - 
specific languages for uhe course-writer, oi creating a file 
directly by using a system which already exists; analysing 
student answers in as much as the dialogue should be as natural 
as possible. (Ultimately this problem is insoluble if a truly 
free dialogue is sought. Since semantics cannot be handled by 
the computer, one lias tc decide to what extent it can be broken 
down into problems of syntax). 

At any rate, when organising all these programmes, a modular 
structure is vital if additions cr modifications ave to be possible 
without upsetting most of the *vrk already accomplished. 



2, SOLUTIO NS FOUND 

A. T he research team 

One of the basic problems was to create a smooth-running team 
made up of both the people brought in to sol'^e technical problems and 
those who would devote themselves more particularly to working out the 
questionnaires. 

The desirable compromise between instructional needs and techno- 
logical limitations requires thorough-going communication between the 
system's users (teachers) and the computer exparts* If this dialogue 
is to be fruitful* at least part of the software group must have a 
solid scientific education in the field involved (in our case, physics). 
For this reason, we started off by setting up two groups: 

The computer group , which war, made up of two faculty members 
teaching the undergraduate electricity course who knew about computers 
and electronics, plus one programme analyst. The purpose of this group 
was to look into the issues raised by the teachers and to come up with 
solutions which would prove satisfactory for the users and compatible 
with the system. 



The instructional group , which wa3 composed of 6 full-time^ 
faculty members, aided by other teachers who were paid extra for their 
overtime work* This group was responsible for drawing up the question- 
naires, on which the experiment rested. 

The staffs and size of these groups f.luctuated from time to time, 
either because people were reassigned to other departmental jtibs or 
because they left to carry on other activities. 

Prom the outset, we encountered various problems in psychology 
which led us to hire a psychologist, who was responsible for studying 
how and what motivated students to learn. 

At the present time, the University Chancellor and the Physics 
Committee have agreed to let the Student Computer laboratory team be 
made up of faculty members, most of whose teaching programmes are 
connected to this experiment, and some of whom are conducting research 
in this area. 



This team was joined by a group of students chosen amongst those 
who had been involved in the experiment. The student groups, working 
under faculty supervision, take part in formulating, writing and test- 
ing the questionnaires. 

In addition, another group of 6 students took on the job of typing 
up and recording the questionnaires. 

The original team of faculty members was made up . ntirely of 
physicists because the first goal set up was writing questionnaires 
for the old undergraduate major course in electricity. Very quickly, 
however, some of our colleagues who were interested in the experiment 
resolved to use the existing system to find out how far it could be 
applied to their own fields. As a result, at the present time, there 
are two teams working in the laboratory, one of biologists, the other 
of linguists studying English. Although these two teams are not 
connected as far as thejr teaching is concerned, they contribute to 
working out new structures, and they take pnrt in overall discussions 
seeking ways to make use of computers in teaching, 

A glance back over the work which has been accomplished since 
this experiment begem and which is summarised in its up-to-date state 
in this report allows one to draw a few conclusions about how to form 
teams to sot up and make use of a computer and its component eouipment, 
in the context of a traditional teaching programme. 

In order for su^h an experiment to be implemented sensibly, a 
minimum staff(2J must be brought together, for otherwise it would be 



(1) T.’iis means that their entire teaching p~o£,ramme was devoted to the 
Student Computer (O.P.E,) laboratory. 

(2) Note that we are considering here a situation in which no prior 
set up exists which could be used for teaching purposes. This 
influences the eize of the otaff needed, especially the number and 
the training of ths computer experts required. 
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foolhardy to expect this kind of undertaking to be successful. Our 
personal* observations make us thin* that the optimum number of staff 
members is about twenty* divided T ntc- 3 groups. 

- A group of computer ex certs of from 5 to 7 members, of whom 
2 or 3 should be analysts and 2 or 3 study programmers, 

- A feroup of faculty members , about 1C in number, including 2 

or 3 who should bridge the gap between the teaching and computer 
staffs, or between the teachers and psychologists, 

- A group of csychologist s , 2 or 3 in number, at the level of 
advanced graduate students, who are able to follow the subject 
matter being studied* since their work must be part of the team 
study of cognitive processes. 

It would be- desirable to have one or two experienced researchers 
to co-ordinate the various activities of such a team, men with broad 
backgrounds, having not only a solid education in science but also 
basic training in psychology and computer technology. 



B, Solutions in ti aching 

Among the many possible solutions found for various problems 
which were anticipated tcf. 1.1, A) we decided to devote ourselves to 
devising a system for self-examination. 

This choice implies the following prerequisite conditions: 

lat - A scientific subject must be chosen, in this case, Physics, 

2nd - A questionnaire must be worked out which has specific purposes: 

- to check up on concepts 

- to supply extra information if need be 

- to make a synthesis of various concepts 

- to provide practice in applying the material 

3rd - The questionnaires must lead to systematic criticism of 
instruction 

- the necessity of providing a chance for unstructured answers 

- the necessity of being able to follow the thought patterns of 
each etuden“. 

- the necessity of recording unexpected answers 

- the necessity of allowing students to make comments, for 
example, about how sure they are of an answer or how difficult 

' . they find the question. 

Hence we ended up by defining a normal sequen ce which includes 
the questions designed to take care of the basic concepts in the part 
of the course involved* and a drifting sequence containing extra 
information designed to enable the student having difficulty to grasp 
some of the concepts escaping him. 



Therefore s we started off by choosing the structure which 
corresponds to the following diagram, in which each question and its 
related answer block is represented by circles, while the various 
branching possible after a given answer are represented by arrows. 

Of course, auy of the possible jumping patterns can be used 
within a single answer-block, since the block allows for various 
processing alternatives depending on the various types of answers 
which have been planned for. 

Later on in this report, in the chapter on applying the question- 
naires, examples are given to show how this diagram was put into 
practice and what solutions were found to satisfy the various pre- 
requisite conditions enumerated above* 



C* Technical soluxions 

After studying the models on the market, we decided to get 
an IBM 360/30 system, whose exact layout is given in Figure 1. 

At the time we were planning the experiment, our budget for the 
project and !;he instructional goals we had set up left us practically 
no freedom of choice. 

It was understood, moreover, that the uroject would get under way 
with a minimum set up> but the aevelopment phase could be undertaken 
without extending the central store and increasing the number of disk- 
pack units. *Vith this in mind, we added another diskpack 2311 to our 
system, 



At the present time, the processor's central memory contains 16K 
eight bit bytes, but we are hoping that we Wxll soon move up to the 
64K because the system is not trustworthy enough. As a matter of lact, 
in order to save on space, we have been forced to cut back on certain 
"hardware" inspection sub-programmes and to simplify our basic system. 
This may be a relatively minor aspect of the beginning phase, where 
the main point is to check that the processing algorithms ere relevant, 
but in the operationc phase it acquires vital importance and causes 
considerable difficulty. 

The diskpack 2311, connected to its control unit 2841, ha3 a 
capacity of 7.5 x IQ 6 characters. It is used to back up the system 
and application programmes, as well as the card files used fer examin- 
ations (questionnaire files, student files) or for the system (binary 
programme files, macro-instruction files, etc,). 

v 

Access-time is relatively short (75 ms, on the average) and 
recordings are organised in the following simple manner: 

Ve have 200 cylindrical drums available, each of which has 10 
tracks. The desired number of recordings can be created on each track, 
within the maximum capacity of 3600 eight bit bytes for each track, 
and provided each recording is separated from the following one by a 
large "between-recording" break 9pace which cute down on the "useful" 
capacity of the track. 








Normal 

Sequence 



O 

ERIC 



- 14 - 



mmtk 






FIGURE 1 



MACHINE SETUP 



Per i o-Reader Printer 





- 15 - 

.<*17 



The 2701-3965 system, which is connected to the multiple channel, 
handles the terminals (model 1052 typewriters). Transmission between 
the central unit and the 2701 is done in parallel (8 bits), while that 
from the 2701 to the 3965 is done in series by coaxial cable. The 3965 
unit is a terminal concentrator which for each terminal has a buffer 
store with 59 characters. This buffer, built with two delay lines, has 
the same input-output for any given terminal. This set up is a source 
of trouble between information coming from the central unit and that 
coming from a terminal (user). The system is built n give the user 
priority* 

All the terminals thus seem to the central unit to be a single 
input-output unit in the form of the 3965, since the distinctions among 
the terminals is made only at the level of the 3965. 

Generally speaking, our system belongs to the co-called "question 
3 vstem" category (rather than the "interruption system" type) in that 
i\ needs to have a programme which systematically questions the 
terminals (the 3965, in our case) to find out if there is any inform- 
ation to transmit to the central unit. 

The 1052 terminals have been modified (see Figure 2) so that the 
speci-'il characters used in physics, such as certain Greek letters, 
indices and exponents, were available# 
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... II. MA TERIALS USED 

In order to implement and take advantage of the questionnaires, 
a certain number of tool3, or programmes were needed, which we shall 
now describe from two complementary vantage points t 

1. The user's point of view. 

2. Row these materials fit into the software system. 

1, THE. USER 'S POINT OF VIEW 

A* Author language - (L t A.) 



We thought it was very desirable for the author of a question- 
naire to be able to get an accurate idea of what an examination session 
was like. The instructional programmes must therefore be such thu; they 
can be introduced into the machine by the author himself, thereby 
avoiding the danger of their being altered by a technician acting as 
middle-man. Since the faculty members generally had had no previous 
training in computer technology, they needed to have a very simple 
method for translating a questionnaire into computer language. The 
Author Language was invented with this in mind. 

The A.L. itself divides the questionnaire into question blocks 
and their corresponding answer blocks. 

1. Question block 

It Is basically made up of the text of the question, within 
the framework of some code characters vl): 

identifier address 



question ■ 

■ . I -j . . > ; - r * 4 . j 

^ 12 ! • * • text ..... 



t 




— question number 

i c r • • r ■* : 



mnemonic question 

rtf! i L . 



end of text — 
mnemonic answer 
answer number — 



answer 




R 12 

^ A 



options 



L asking how 
sure student 
is 

type of 
answer 



- 19 - 

is: 

20 



(l) R o Answer. J = Right. 
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The 4 * R 12 characters are redundant but convenient for process- 
ing. There are two options: 

- the first character is J or + depending on whether the expected 
answer is of the wo rd type or formula t'^ne , The student's answer is 
analysed by the programme chosen accordingly (see Verbal Analysis and 
Formula Analysis ) • 

- the second option character makes it possible (+ code) or not 
(- code) to ask the student to show how sure he is of his answer (a 
number from 1 to 5). This result is used for research in psychology, 

2# Answer block 

It is made up of a group of typical answers (RT), each of 
which is accompanied by some remarks and followed by the number of 
the next question. 

The text of the RT is made up of one or several "comparandes" , 
that means, the formula or the words with which the comparison is made, 
Next to the remarks matching each TA, a symbol is put down to indicate 
whether the answer is right (J) or wrong (F), Furthermore, an option 
character makes the work of analysing the answer more specific. 



For a formula: + or - 

For words: 0, D, vu or 3 (see analyses). 

In other words:. <•, • 

•RT, eomparande • • • > 



mnemonic 
typical answer 



mnemonic 



remarke 



right - 



C J + 



text 



option analysis . 



iSLMj 



where the next 



enl of text U -gS“S» iS 






I- 



The author can anticipate as many as 9 different typical answers 
(RT), which are examined in turn during the question period. Unless 
they match perfectly, a standard set of * narks and the matching 
question number make up the line Other Answers . 



AR? 



* 






_ l ! - 

code: mnemonic 



text . . , Q27 



Other Answers which are considered False, 
In this way, each question can be processed in different ways. 






r «s!W 






Remark 



The first typical answer differs slightly from the others, in 
that it serves to identify the Answer Block and as a result, the RT 
becomes E XX (R 17, fox example, where 3.7 is the corresponding question 
number ) • 

r The "comparande" which forms the answer is automatically sent on 
to the student (whenever there is a. loop, that is to say, when the 
question is asked a r^ond time and the student answers it incorrectly 
the second time). The student is then supplied with the correct answer, 
and the examination session continues with the next question. 

As the system was worked out, different branching possibilities 
(cf. I.2.B.) were modified for various pedagogical reasons. 

Going off one question forever and on to the next is a move put 
in the category of ' r other questionnaires* • This act is accomplished 
by filling out the aduress of the question which shows up after the 



remarks. 

CJ+ ........ text . . . . .*Q17, for example, becomes 

CJ+ • • . ... . , text . • . • • *K28~Q17, if the student wants 



to get hooked up with question No. 17 on questionnaire No. 28. 

There is smother way to digress to another questionnaire (cf. 
Associated Description Language ...). 

The possibility of going back to questions asked earlier is the 
normal sequence, which was very rarely used, has been eliminated. 
Indeed, it seems preferable to find a new way to express a concept 
which was not grasped well the first time through. 

The idea of a normal sequence was relaxed, in that side jumps no 
longer have to lead either back to their starting point or directly 
to the next question. This limitation had originally been introduced 
to make work easier for the writers of the questionnaires by providing 
them with a systematic operating flow chart. 



B. Associated description language (L.D.A.) 

The Author Language has two main disadvantages: 

- sequential structure in numbering the questions. 

- branching is determined by the immediately previous answer. 

The first characteristic makes each question practically a 
prisoner of its questionnaire. Jumping from one questionnaire to another 
during the course of an examination session causes definite complications, 
and consequently must not be done too often. 

The second characteristic is a pedagogical limitation which can 
make it impossible to follow the individual student's thought processes 
during a session. * . * * ' 
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Therefore, W3 decided to write out another language, to be intro- 
duced at a second level, to make up for the shortcomings of the basic 
language. 

The L.A, remains, and continues to be the only wa> to feed texts 
into the machine. But those professors who have acquired a certain 
mastery of siting out questionnaires have the L.D.A. at their disposal. 
This "associated" language makes it feasible to create a new examination 
starting with the already extant questionnaires. All they need to do 
is to decide on the contents and then set up once again the sequence 
to be followed. 

1. Sym bols used 

A question is referred to by the mnemonic symbol Q, followed in 
parentheses by the questionnaire number and the question number# For 
example, Q (18,12) stands for question 12 on questionnaire 18. Follow- 
up questions (whatever answers are given) are written out in simple 
chain fashion, separated by commas: 

... Q (18,12), Q (11,27), Q (45,03) ... 

The examination session takes place in the fol?.wing pattern: 

- the students answer is normally compared with the different 
Typical Answers contained in Answer Block 12 of questionnaire 
18. : , ■ • 

- the connected remarks are referred on to the RT which matches. 

- the number of the question given at the end of these remarks 
is skinned over ; instead, the session moves on to the text of 
question 27 on questionnaire 11. 

The same process is repeated in this manner after each answer# 

This requires the order the questions are in to b9 independent of the 
student's answers, and that is very rarely the case. Several alternative 
branching orders generally must be planned# The question must then be 
defined# Here are the symbols adopted for one specific cases 

Q (17,14) ^ 1,4,Q(36,11), Q(42,15) 

2,3,Q(54,2l) 

5,S,7,Q(28,07) 

Interpret at ion 

* each line defines a particular branching. 

- the figures immediately to the right of the = sign stand for 
the different RT numbered in order starting with 1. 

- after the list of figures comes the next question to be asked, 

rr - followed perhaps by other questions# ' • - * 

- the lafct question, given on the right of each line, will be 
defined elsewhere# 
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Ifp for example, the answer the student give'! happens to match 
the second or third RT (of Block 14 on questionnaire 17), the related 
remarks come up, of course, and then comes question .71 on questionnaire 
54; .this question ic then defined just as Q (17,14) was. The " descrip - 
tion of an examination session is hence made by means of a set of 
" def inition s". and each question which appears on the far right of a 
line must also show up on' the left of an = sign. The beginning and end 
have special status which will be made more explicit in the example 
given below. 

The material is put into the machine on cards (one cord per line, 
with a lather free pattern). The syntax will be explained later in this 
report. 

2. Extending the language 

- Limited to the above, the L.D.A. already makes it possible to 
rewrite a normal kind of questionnaire, eliminating the ’ restrictions 
imposed by sequential numbering. Now let us look at the new possibilities 

- further remarks 

- indicators (or flip-flop switches) 

- counters 

- tests for the indicators and counters 

(i) Further remarks . The author can stick in extra remarks between 
2 questions; the mnemonic is C, and the text is fed into the 
machine like a sham question followed by a simulated Answer 
Block. The reference number in the card file is the same as 
That for a questions 

C(53,17) 

stands for the text of question 17 on questionnaire 53 (which 
will be given out as a set of remarks, without waiting for 
ai\ answer). These remarks follow the normal remarks connected 
to an RT, It is obviously convenient to group together the 
texts of the remarks and the sham questions, 

(ii) Indicators (or _fllo-f loo switches ) . The author can insert 

"indicators” in a line of description to follow the student's 
thought processes. These are simple flip-flop switches (binary 
position), set at zero at the beginning of the examination 
session, and witch change when they are encountered. The 
mnenomic symbol is B. It is followed by a number which stands 
for ‘-h? number of the flip-flop. 

For example: Q(17,14) = 1,4,B3,Q (36,11)... 

• ♦ t • . 

means that flip-flop No, 3 (set at 0) moves to 1 if the student 
■ answers in manner 1 or 4# B3 can be used again in the descrip- 
tion* if it has already been switched on, a second encounter 
will make it return to .the 0 position, a third will return it 
to 1, and so forth. How the contents of these indicators is 
used will be explained below. 
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(iii) Counters . These counters are stores in which the value judg- 
ments given to the students 1 responses during the examination 
sessions are automatically kept. The mnemonic is CTR, followed 
by a number which identifies the counter or its function. The 
counter CTR 1, for example, has the task of summing up the 
number of correct answers (found in each RT, on account of the 
letter o at the levil of the code for remarks). The contents 
change constantly as the session progresses, and so it can be 
called for at any time, 

A second counter, CTR 2 can be' used to add up answer time, 
and so on. The syntax provides for adding other counters as 
may prove interesting, 

(iv) Branching by means of indicator or c ounter tests . Instead of 
stating the next question right after an RT, the system can 
shunt the student off to an indicator or counter test. This 
makes it possible to orient him (at leaBt partially) on the 
basis of his earlier thought processes or of the irstant 
va> ae of certain parameters. 

Indicator test . 

Mnemonic TI, followed by a 2-digit number standing for the 
indicator tost number. An example of how this is written: 

Q ( ) = 1.5.Q ( ) 

2,3,TI15 

4.Q 



TI15 - SI B3,1,Q ( ) 

2,Q 

The TI15 test is hence simply substituted for a question, and 
ia defined below in the same way. < • . 

SI B3 showt that the branching depends on the consents of 
flip-flop No. 5. There are two possible patterns: 

B3 is at 0: branching in 1 
B3 is at 1: branching in 2 . 

The text can deal with several flip-flops at the same time 
but can lead to only one branching choice: 

1: at least one flip-flop is at 0 
2: all f lip-flops are at 1. 

The counter cest (mnemonic TC) is used in a way that is quite 
similar. The 12th counter test, for example, would be given by 

• TC12 « SI CTR I - (8),1,U( )•>; 

2,Q( 

' 3, Q(. 




- 24 - 








In this case, the test deals with counter No * i (which adds 
up the correct answers). Its contents are compared to the 
number between parentheses. The branching depends upc;> the 
outcome: 

If the contents aie greater than 8: branching in 1 

If the contents are equal to 8 : branching in 2 

If the contents are less than 8 : branclang in 3, 

3, ft concre te example of use 

The following description provides the non-scientist reader with 
a complete illustration of the many possibilities. That all the texts 
belong to the same questionnaire results solely from the fact that the 
subject involved has not been worked out in other questionnaires* 

The material below consists of: 

- a description of a questionnaire (No ,15) written :Vn P *D,L. 

- an excerpt from the same questionnaire , 



DESCRIPTION OF QUgSglO MjAIRg 15 IN A.P.L 



0010 

0020 




S=Q(15,l) 



0030 2 
0040 I 



0050 1 
0060 1 
0070 2 



0080 



0090 1 

0100 1 
0110 2 
0120 < 



> 4 r QU5f 4} 

Q(15,7)=1,2,4,Q(15,8) 

! 3 f Bl,Q(15,8) 

Q(15,6)=B1,TI01 

:), 0(15,11) 

i 



0130 2 
0140 I 





0180 2 
0190 i 



0200 2 
0210 



0220 1 
0230 2 



0240 Bin) 
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SOME EXCERPTS FROM QUESTIONNAIRE 15 
: QUESTIONNAIRE 1-j 

K15 QUESTION 

HERE ARE SOME UNRELATED QUESTIONS, FOR THE PURPOSE OF SHOWING 
HOW CONVERSATION WITH THE COMPUTER WORKS. ARE YOU READY? 

Rjrfla+ 

XKXXXXXXXKXKXXXXXXXX 

K15 ANSWER 

1 YES •• 

CJ+ PERFECT 

Q<51 

xxxxj* 

2 NO 

CJ+ WE SHALL BEGIN ANYWAY 

■ YOU HAVE NO TIME LIMIT FOR ANSWERING 

Q/l , 

xxxxx 

3 

ARF+ KINDLY ANSWER MORE CLEARLY, PLEASE 

QA ••• - ••••'■ 

XXXKKKXXXXXXKKKXXXXK 

fiQ K 15 QUESTION <$2 

*HO WAS THE FIRST MAN TO FLY 
R<S2a+ 

XKXXXXKXXKXXXXKXXXIX 

KL5 ANSWER <52 

1 THE MONTGOLFIER BROTHERS 

CJ+ VERY GOOD ) ' 

Q 62 

xxxxx 

2 THE MONGOLFIER BROTHERS 

CJ+ VERY GOOD 

0^2 

xxxxx 

3 MONTGOLFIER 

CJ+ GOOD 

q <$2 ' 

xxxxx 

4 MONGOLFIER 

CJ+ GOOD 

Qt2 

xxxxx . - , ; 

5 ADER 

CF+ NO. CLH4ENT ADER WAS ONLY THE FIRST TO FLY IN A "HEAVIER 

THAN AIR" MACHINB. 

THINK HARD AND 01 VH ANOTHER ANSWER. 

Qrf2 ■ 

xxxxx 

6 ICARUS 

0F4 NO, THAT 13 ONLY A MYTH. QIVE ANOTHER ANSWER. 



Q*te 

XSl'SV 

7 I DON'T KNOW 

CP+ YES, OP COURSE YOU DO 

cute 



9EMHX 

8 




XJ.XXXXJMMCXXJOMUt 

/8 KL5 QUESTION fo 

BE CAREFUL ABOUT THE SPELLING, MONTGOLFIER TAKES AT ' 

• • ; Rfca+ • - _ • I ; 

X*X****KXK**X*K 

. K15 , liUBSHCN # 

1 .. - " ■ ' ■ — * • ■ - 

CJ+ •• 

q^ 3 - ; . : / , / ' \ > ; r J : . - ■; ■ ... . 

***** 

2 ' ■ * 1 ' ; • : : ' 

aKP+ ' '' 5 • ' ’ 

, o^3 ; • 

XSKXSXXJfXKSKXlM ... 

i 8 K15 • QUBSTION M ' 

HERE IS A HINT TO HELP YOU. THE NAME IS STILL USED FOR THB 
TYPE OF DEVICE INVOLVED. . ■. 

TRY IQ ANSWER ONCE MORB. 

: R|*4a+ V • ' ' 

XXXRXXXXXXKXXXK 

K15 ANSWBR M 

1 MONTGOLFIER - - ; • : 

C.I-H GOOD 

Q*4 

MW* 

2 MONGO LFIER • 

CJ+ GOOD 

QM 

XX XXX 

3 . MONTGOLFIERS - balloon 

CJ+ NO, IN FACT THE FRBNCH WORD FOR THIS DEVICE IS SLIGHTLY ' 

' DIFFERENT FROM THE NAMB OF THB INVENTORS. 

Q^4 ‘ 1 

xnoa 

4 . 

AHF+ r ‘: NO, THINK HARD. IT INVOLVES THE EARLIEST HOT-AIR BALLOONS. 

' . 7' TRY ONB MORE TME. \. v : ", ” *•; * 

<u* 4 •. • " “ • " v •; • • j 

SXXMUWKJUUX1UUI 





4. Criticism and outlook 



The idea of L«D.A. came into being at a time when we had not 
examined the notion of normal sequence. Language has a structure based 
on this very notion, and that brings about the following limitation 
{already mentioned from the original L.A. ): 

- any shunting off to a side question (whatever the level) must 
end up back at the original question or at the next question. 

It so happens that this requirement is paralysing, when it comes 
to using the tests. As a result, it is difficult to use L.D.A., and 
that has been done very little up to now. Furthermore , only questions 
which are sufficiently separate from the context of a questionnaire 
can be used a second time in a description. At any rate, many extra 
commentaries seem to be required, . but at the present time, it is not 
possible to string out several commentaries in a row* 

It should be noted, however, that all these 4 limitations come from 
the fact that the central store is small. 

The current L.D*A, is hence only one stage in the development of 
a much more flexible language, and plans are now underway to make the 
major changes which are absolutely necessary. Nevertheless, the tests 
done up till now are promising, and have served to bring out clearly 
what are the desirable characteristics of a second-level language which 
will really perform correctly. Such a language will provide an instructor 
who is already familiar with how to write up questionnaires in L.A* with 
a tooJ particularly well designee to meet teaching needs. 



C, How answers are analysed 

An answer can be processed in accordance with any of the following 
three alternatives: 

- without analysis 

- with verbal analysis 

- with formula analysis 

(The type of processing chosen depends on which staff does the 
writing), 

1, Without analysis 

When this alternative is chosen, the series of characters which - 
makes up the answer is compared directly (rough comparison) with the 
scries of characters which was planned for' by the author behind the 
RT code. 

If the two are the same, the commentary is transmitted as planned 
to the terminal and branching occurs onto the question number following 
the commentary; otherwise, it goes on to the next RT, and so on until 
the AR? code is reached. 





2. Verbal analysis 



This analysis carries the code J in the question block, (cf, 
description of the L.A,). 

(i) Purpose . Do seek out in the Student Answers (RE) those 
expressions which appear in the Standard Answer (RT). 

"Expression" means consecutive words , where a word itself 
is made up of any kind of characters except those reserved 
to define "word limits'* such as the blank space, the comma, 
etc# (see, below at options). 

Some examples of expressions: 

- magnetic induction - 

the little cat likes milk ‘ 

- tensor ■ - ! ' . • 1 ■ 

- sodium acid sulphite 

- K - 23 -fl 

The last of these is a formula, but in certain cases it 
may prove convenient to treat it like an ordinary expression, 

- To detect negatives, RE and RT are considered to be the 
same when all the expressions requested have been found and 
when no negative has bee 21 uncovered, except if the answer 
is in the negative form. The other words which may be in 
the RE are overlooked. 



(ii) Filling out a Typical Answe r* The comparison which the 
author requires can have bearing on several expressions 
at the same timej in addition, each expression can assume 
equivalent forms (synonyms). 

Let A, B, C be three expressions asked for, each having 
equivalents in A', B 1 stud B f, f O'. It must be possible, 
therefore, to find in the RE a way tc express each of the 
') : ; 3 following columns: 

A B C 

■ - ^ A 1 b 1 - C* 

B 1 ' 

For setting up the corresponding RT the author lays out 
- - the 3 lists end to end in a straight line. He only has to 
, - put the 1 — — t sign (tabulator key on, the terminal) between 
the lists, and the // sign (2 slash s) between the 
equivalents. j - 

, In other words; r . ^ ; 

RT A//a'> B//B'//B" I -i C//C 1 



This way of writing it is logically equivalent to: 

(A or A 1 ) and (B or B r or B 1 1 ) and (C or C') 

(iii) Options . Two kinds of independent options are available. 
They involve: 

a) the order 

b) the characters for separating words in the RE 

When several different expressions do indeed occur, the 
order in which they appear in the RT may or may not 
determine the quality of the answer. Furthermore, it is 
Immaterial in French how words are surrounded by blank 
spaces, commas, periods, and apostrophes. In the case of 
mathematical “words" » such aa R = 2,3 A or A.B, or even 
L)'=D, the decimal point, scalar product and the prime sign 
are normal characters. Only the blank apace continues to 
serve as a way to separate words. Consequently, on the 
level of the commentary connected to each RT, the author 
must use a letter to specify which option he has chosen. 
One of the 4 codes given in the latticework which follows, 
which summarises the 4 possible combinations, is put down 
after the CJ (or CF): 





Order 


Disorder 


blank space . , 9 


0 


D 


complete blar?k space 


u> 





(iv) Examples 

a) Ql% In order of decreasing electronegativity, state 
the names of the different halogens R12J+ 

R12 They are fluorine, chlorine, bromine and iodine 
CJO *Q13 

RT FLUORINE CHORINE , . BROMINE » 

IODINE 

CJO Very good. *Q13 

RT IODINE CHLORINE BROMINE i— i FLUORINE 

CJD Yes, bgt you gave them in the wrong order. Answer 
again Q12 

b) QXL What is the resistance R and the intensity I 

, flowing there *R1?J+ 

R17 R=2,3fl//R=2.3 0HM3 1*0,4 A//l*0.4 AMPERE 
CJJ flood *Q10 
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3» Formula analysis 

Whenever the answer appears in formula form, it is impossible, 
first of all, to anticipate all the equivalent' ways of writing.it, 
and secondly, it is not desirable to introduce very rigid limitations 
on ways of writing, such as fixing the order in which variables are 
given, positioning parentheses and quotients, ar-d so forth, which would 
tend to make using the system cumbersome. ' 

As a result, it became necessary, to plan a programme which, to 
a certain extent, would analyse the students answer, the one expected 
by the staff that wrote the programme, and Which would then decide 
whether they amount to the same thing. 

From a theoretical viewpoint, this problem is a rather complex 
one, so that we aha +1 consider here only the aspect of it which deals 
with "practical applications". 

A syntax for the formulae used is worked out. This syntax is 
described in the programme by "context-f ree” rulss of the type 

<formula> -*(relationship> < formula end characters 

<relationship> -Xside of equation > relationship operating signs 

" ' * ' v ‘ <8ide of equations 

^relationship operating sign>*=, SUP, IMP 
• ' <8ide of equation terms 

reside of equation>i <side of equations 
-><term > / <term> 

etc ... 

In practijal application these rules will fix itie restrictions 
set cm* programme writing, which in turn. will, define the limite of the 
Operating signs and the relative substitutebility of- the different 
symbol e 1 appendix on "Restrictions on programme writing 1 ). 

> These rules enable us to analyse a formula at the input point 
and to decide whether or not it belongs to the L(G) language which 
we have defined, that is , whether or not it is syntactically well 
formulated. It is worth noting, however, that the syntax was defined 
Only for a subset of the set of possible formulae, so that certain 
cases will not bp analysed (expressions containing a vectorial product 
or an exponential equation), > * ; 

‘ If a "mistake in syntax" occutb, a message is sent to the student, 
vhd can then suggest'- another answer, for it is important not to penalise 
a careless error or a mistake in typing** 

Once the formula is syntaotically acceptable, it is put in 
conventional form so that it can be compared to the standard answer 
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(which is also stated in conventional form when the questionnaire is 
entered on diskpacks). - 

The process consists of rewriting every Bingle relationship in 
the form of a series of signed teres, such that the second side of the 
equation equals zero. It is generally made up of 

- expanding the parentheses 

- reducing to the same common denominator 

- rearranging factors and terms in a given order 
. - simplifying (in certain special cases) 

The comparison results in dividing things in two. One does not 
strive to fix semantic meaning on missing items, If the two do not match 
perfectly, the answer is considered false and is processed accordingly. 



RrampT e « 




; \ “ ' ■ • • v ! ,-: . ‘ h. 

Typical answer , •*, ;; 


J ROT H = i + t J t S 


Conventional form 


■ i -t -c 0 a t * ' ROT fi = 0 


Student's answer 


• t ■ •• (ROT H)/ 1 0 = V t Q + ^ 


Conventional form 


: > : -1- e 0 a t 1 + ROT ft = 0 


Judgment 


, : Correct 



Moving from one standard answer to the next is done in the same 
manner as In processing without analysis (CP 1I-IC1). Moreover, this 
is true no matter what kind of analysis is asked for. * 



D. PCM - Information system during Questioning 
1. Descripti on 

! v The goal which was set. up was purposely limited to allowing the 
student to have access to definitions coming from a recorded dictionary, 
while the questionnaire is being used. This dictionary is input by the 
authors according to the needs they anticipate for each questionnaire. 
Under present conditions there is no protective device, so that students 
have the entire dictionary at their disposal. Prom an instructional 
point of view, making info mat ion available to the student forces him 
to be more aotive and stimulates snatches of dialogue which replace 
the habitual "tutorial" method of teaching. That reflects a need vhich 
the authors and the students often feel. In addition, the questionnai :es 
can partly be relieved of their "introductions" which are often long 
fluid tedious. The system is not fixed once and. for all - improvements 
are planned. The description which follows shows the present experimental 
stage of development. It enumerates the various operating etepsi steps 
the student takes to get information, and steps the Authors take to 
supply and keep the file of definitions up to date. 



2# Definition keys 



All s tudent or author operating steps must include a definition 
key which may be: - : 



1. An isolated word {Ex'unple: PHASE) 

2. An expression of the "Law of 11 type: (Example: LAW OP PHASES) 
The expressions currently allowed have the form: 



f LAW{S) 

l E^UATIONiS) j 

I THBOREM(S) A 

RULE(S) ' 1 

'RELATIONSHIP(S), 




3. Any sort of expression whatever {the difference being that 
in 2 condensation is involved). 



The definition check may include as many blank spaces as 
(they are later eliminated)* Back-spacing in order to make cor *r u 
is allowed in the key and the definition* 

The length of the keys is not limited, nevertheless, it if. 
important to know that some truncation cay occur after condense - 

3. Capacity 

The length of the definitions is purposely limited to abc t 
hundred characters# A gimmick can be used for longer dof initio : 
are defined' in a chain, in which the first definition contains >■ 
which is defined elsewhere, and so on. The fil^ currently bein 
can contain 1500 definitions, 

• 4# Student operating control 

Each time It is the Student's turn to take over, he can cal _ 
information in the following manner: 

*m - Key . . s ; . . . 

For example: *DCM *PHASiS (as many blank spaces as are needed) 

He receives the definition or the. message NO INFORMATION. ^ 
then continue the questionnaire he is in the midst of doing by a' 
ing the question which was asked, or else he can make the next si 
operating move, u ;■ \ 

- felling mistakes are not accepted. This limitation is by 
unreasonable, since the words whote definitions w re being aske 
usually printed in the text of the questionnaire. 



O 

RJC 
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5 • Author operating control 

- Cataloguing : This operating move takes the form: 

*DCM* ©Key Password = Definition 

Example! 

DCM PHASE = HOMOGENEOUS PART OF A SYSTEM 

Altogether it must not require more than two blocks (120 characters). 

The system sends the message DONE if the definition is a new one, 
otherwise REVIEW DONE. 

- Erasing : Erasure is accomplished in the following way: 

*DCM *DE.LETB Key 

Exaaple: *DCM *D£LETB PHASE 

The system sends the answer DONE or a message ’R r if there was 
no purpose to the operating move, 

Hpten The Author Operating Control moves can be executed during the 
session, 

6 , Single- character messages 

The system sends these messages to signal that something has 
gone wrong: 

oo wrong operating move (the three-letter operating code is not 
DCM, nor any of those recognised by the system) 

x the asterisk at the end of the definition is missing 
= the equals sign at the beginning of the definition is missing 

The other messages are for use within the system. For example, 

P indicates that both the available track and its corresponding emer- 
gency track are full* 

7, Inventory of recorded definitions 

Au inventory programme for definitions or a fil9 is available 
for the authors. It is called for from the computer console. Its name 
is LST DOC. " ; : ; 

_ 8, Recording requests for information 

During a session, a special command desk - usually the instructor's 
desk - makes up a list of the requests for information (definition 
keys) which the students have made, placing 4 DCM in front if these 
requests have been satisfied, and -DCM if they have not been. There 
are several instructional advantages to this; 
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- A statistical study of requests for information 

- Improving the information file 

- Improving the questionnaire by adding to it the frequently 
:*sked for definitions 

- Statistical evaluation of the students* level and the gaps 
in theii 1 knowledge. 

Probably some of these interactions will be made automatic. That 
would represent the next step in progress towards a freer dialogue 
between the student and the Computer - aided Instructional System, 



E. Programmes in use 



1. programme 

Description of how it is used. This programme comes into play 
when a questionnaire is typed out on a deck of cards. It uses the deck 
to supply t*'o types of information, 

- Operating programme : An operating chart giving the logical 

form the questionnaire should take is sketched out by means of a rapid 
printer. . • 

- Checking the syntax: Vhile the operating programme is being 

printed, the deck of cards is checked out so that it agrees with the 
syntax of the Author Language. If there is a mistake, an accurate 
prediction of the card involved and the column number on that card are 
provided. Besides actual errors in syntax (invalid codes or codes in 
the wrong order), other common mistakes (for example, finding a card 
which does not belong in this deck) are spotted* 

This programme is a vital step between typing or key-punching 
the questionnaire and feeding it on to discs. Indeed, putting a 
questionnaire on discs which has the wrong syntax sometimes brings 
about unexpected results. 

2. Punching t)y? & 

, The input procedure for a questionnaire written in Author Language 
includes the intermediary step of establishing a deck of punched cards. 
Due to the fact that we are using a set of spocial characters, it is 
not possible to perform this operation on key-punchers. Therefore, the 
1052 consoles were used for this purpose. The punching programme 

- manages the terminals 

- checks at every st8p of tue way to see that the question blocks 
and answer blocks are built in accordance with the rules laid 
out by the L.A, 

3. Listing 

Every quest ionnaire put on discs should be able to be listed at 
any time. For the same reasons as given above (the rapid printer is 
equipped with a standard chain), this operation must be done using the 
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1052 console. Whatever questionnaire is used, tbo listing which is 
done contains aach question, followed by its related answer block and 
all the relevant code indications, identifiers, and so on. 

4. Re cording and issuing unexpected answers 

Each time the ARE output of the Answer Block is used (cf. Author 
language II 10), i.e. each time an answer i3 not analysed , it is 
recorded on a disc. One or several of the desks can print out the list 
of unexpected answers at the end of an examination session or workday 
on one or several terminals. This list contains the unexpected answers 
sorted by questionnaire number and question number r as well as by 
registration number in the author r s student-file for each su-.h answer. 

5 . Handling and processing student data 

While an examinacion is going on, the following data is recorded 
for each question answered by each student: 

- the outcome (right or wrong) 

- the amount cf tine the student took to answer 

- how sure the student is of his answer 

By using this data, the thought processes of the student in the 
course of the examination (sequence of questions and answers, outcome) 
can be followed, and statistical records for the questionnaires can 
be set up, 

2, HOW MATERIALS FIT INTO THE SOEItfARS SYSTEM 



A. Basic system 

The software which the constructor delivered includes two 
systems: the Basic Operating System (B.O.S.) and the Bisk Operating 

System (D.O.S,). 

Each of these two systems is based on a disc. Moving from one 
system to another is done by replacing the disc on the 2311 disc unit. 
Different programme e which are in use enable the user to cut down, 
increase or modify the possibilities of a system depending on his 
needs* • / v -' v - s, -‘- •- 

- a u initial loading programme (1P2) 

- r supervisor programme 

... . - inspection and service programmes 
U ■ - compiling programmes 

- commercial programmes 

The supervisor's tasks can be summarised as follows: 

- .* . , 

- processing interruptions 
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- handling input-output requests 

processing programming errors and input-output errors. 

W9 should call special -attention to the "FETCH" supervisor sub- 
programme, which makes it possible to place in the central store and 
carry out immediately any of the programmes recorded on discs in 
untranslatable binary form (CORF IMAGE LIBRARY file). 

The essential difference between the B.O.S* and the P,.Q,S. i 3 
in the nature of the programming languages which can be used with 
each system, . 

The B.O.S. only has an assembler and a compiler for Report 
Programme Generator (RPG) language, which is useful solely for 
commercial applications, - r 

The D.0.3, also has a FORTRAN compiler. However, a system which 
includes a 6K- eight bit byte supervisor and programmes that are 
complicated enough to handle input-outputs on diskpacks demands too 
much central storage space for us to use it in our experiment at the 
present tlr.e. 

It Modifications made in the B,0,S, system 

For want of storage space, we were not able to utilise the D.O.S, 
Furthermore, the B,0,S, as such was not appropriate for our particular 
experiment. We nevertheless preferred to hold on to the B.O.S. and 
modify it for our purposes, rather than to completely rewrite a system 
adapted to our needs. 

These modifications can be put into 2 categories, 

(i) Loc al modifications . The result of these was to clearly 

improve the usefulness of the B.O.S, , especially by increasing 
its performance and by creating system service programmes. 

The modifications consist of: 

- Eliminating reading inspection cards, by means of a programme 
hooked up directly to the control panel. 

- Automatic loading of phases, or of modules recorded on discs, 
available for use immediately > or returned to the programme 
calling in (macro LOAD), This change makes it possible, in 
particular, to keep the system updated, and to adjust 
directly any programme on file in the Core Image Library. 

- Adapting tne D.O.S. macros CALL, SAVE, RETURN for the 
B.O.S, system. 

- Writing a "disassembly” programme or reconstructing an 
assembly progress on the basis of binary codes. This 
programme is especially interesting when it is necessary 
to alter or patch up a "constructor programme" for which 
the listings are not available. 

- Writing service MACROS. 



(ii) Writing a programme to handle messages for the 1052 , Since 
the 3965 is not run by the B.O.S., it is necessary to write 
input-output sub -programme 8 for the 2701-3965 combination 
which are compatible with this system. How this programme is 
hooked up with the application programme is described in the 
next paragraph. 

B. Application Programme 

This is the term which covers all the programmes involved in 
practical applications and which are organised around a resident 
component which we call the "core" • 

This core serves to handle the message input and output lines, 
and to call up from the central memory the various programmes required 
to process a given input message. These programmes are called upon by 
means of the supervisor's FETCH macro which we adjusted in the follow- 
ing way. Instead of reading the entire system dictionary on diskpacks, 
we created a tiny dictionary of the phases used in questioning and we 
put this into the central store. This modification makes it possible 
to avoid long and useless disc readings. 

The core* s structure is giver* in Figure 3* The non-permanent 
programmes are brought to a relay area which is described in Figure 4 f 
which shows how the system is input into the store. 
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FIGURE 3 
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Ill, APPLICATIONS 



At the present time* the experiment has expanded to include 
teaching staffs in a number of fields, 

1st year M, A, level 
Undergraduate major 
1st year M.A. level 

Beginning undergraduate level 
for physics and chemistry majors 

In addition, the Student Computer Laboratory (OPB) has begun to 
work with some professors. teaching Physics to undergraduates. This 
co-operation ought to grow this year, since four professors have been 
granted temporary half-time assignments to our staff by the adminis- 
trative authorities in charge of instruction and school programmes. 



1, HOW THE PROCRAMKh WAS APPLIED TO PHfSIC S 

The library of questionnaires on the physics course at present 
contains 35 questionnaires on the syllabi of courses leading to the 
M, A, degree in physics, 

As will appear from what follows, this experiment lent itself to 
other fields, such as biology, English-language linguistics, and 
classroom discussion work in numerical analysis. In this chapter, 
however, we will ojily deal with what is lelevaat to the problems that 
cropped up in the course of work on the physics questionnaires and the 
materials which the team of computer experts cams up with to help the 
instructional staff solve them. 



„ - Physics 

- Biology 

- English-language 
linguistics 

- Discussion classes in 
. Calculus 



A, Eerf eoll ok the questionnaires . 

Once the subject matter was selected and delineated during the 
meetings which the instructional staff had with the professor in charge 
of the main lecture course, we found we were dealing with some 25 to /, \ 
30 unite covering the material of the undergraduate Electricity course; 1 ^ 



(l) Aftei the M.A, programme in Physics was reorganised into four major 
courses (Certificates 1,2,3 and 4) with syllabi different from those 
of the old undergraduate physic s programme, we broadened our field, but 
the methdds used to debug the -questionnaires developed very little. At 
the present time, we are trying to systematise the way the questionn- 
aires are written by using the help of a team of psychologists. However, 
this aspect of the project is barely under way. 



To begin with, each of these units was entrusted to a team of two 
teachers who were to take charge of the resulting questior laire. The 
first step was then to work up an outline for the questionnaire, which 
was then discussed by the full teaching staff. After this initial 
discussion, a temporary draft taking the remarks made into account was 
made, typed up and fed into the machine. 

At a full test meeting of the staff, the questionnaire was then 
reworked, criticised for both form and content, sometimes flipped 
completely around, and always filled out and made more accurate, 

Stuaent membership in the groups working to write the questionnaires 
led us to chang 3 our methods slightly, having each group keep up with 
its questionnaire until final testing, though some members of other 
groups would also be involved by that stage* 

Once the subject is chosen, the questionnaire is thrashed out with 
the following moves: 

- Deciding what kind of questionnaire: 

- Questionnaire in which emphasis is put merely on testing. 

- Questionnaire in which the goal is to get a law stated, or 
more exactly, its symbolic formulation. 

- Questionnaire in which only the basic concepts are made 
explicit. 

- Deciding on the normal sequence and the order in which the 

steps should be taken* 

Out of all the notions which come up during a study of the 
subject chosen, which ones are directly connected to reasoning 
and thereby merit a place in the normal sequence, 

- Deciding on the questions for a derived sequence. 

Starting with the outcome of the previous choices, their 
formulation in question form, and the standard type answers 
that they call for, one can move on to building up questions 
in derived sequence. 

B, Writing the questionnaires ’* / 



. The following points have been emphasised: 

(i) Need to establish the preoise teaching goal of each 
questionnaire* In particular, how does the teaching 
occurring here fit in with the lecture course and the 
discussion sections, . . • . • 

Practical example worked outi concentrating on some basic 
notions which often prove difficult for students to handle 
(in the questionnaire which was used, concepts of macro- 
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scopic and microscopic states and their relationship to 
concepts of probability were involved). 

(ii) Need to establish the goals expected from anal ysing the 

results . These goals can vary: feedback on the course or on 
the discussion sections, analysis of how concepts are learned, 
relationships between concepts, student thought processes, 
causes of error, etc. It is difficult to reconcile all these 
goals, and the questionnaires do not necessarily take the 
same form for various situations, so that it is necessary to 
select some of them beforehand as the chief goals, 

(iii) In every case the questionnaire must be designed to satisfy 

these purposes as much as possible. That assumes that experi- 
ments will be used to check up on how adequately the rules 
for design match the goals which have been established. Those 
writing the questionnaires are obviously intimately involved 
in this aspect of the work. 



2. Method used 



The questionnaire deals with the binomial law - it aims at 
making the student go over some notions (cf. III-B-l-i), and attempts 
to analyse the largest possible number of probable errors and their 
causes, and to rectify the situation. Hence, it is designed to make 
an initial breakdown of errors. Drawing up the questionnaires was done 
in several steps. * 

(i) First draft , written in the style in which it is given to the 
students. The goal was to see how big the subject is, how it 
caa be outlined, how much data need be presented, how long 
it should be, etc. 

(ii) Analysing this first draft . The purpose of this analysis is 
to pick out: 

- the notions introduced in the statement 

- the notions which the student is assumed to know 

- an analysis of the form the questions take: what concepts 
or string of concepts are they based on? 

- cm analysis of the reasoning process required to move from 
the question to its answer: locating all the steps which 
the student must take to rnswer correctly (obviously the 
level on which this analysis is made must be related to 

■- 4 how much the student is assumed to know. The level is 

correct when no mistakes show up on material of lower levels). 

- the foregoing analysis should make it possible to sift out 
, beforehand certain kinds of errors and their causes. It is 

then feasible to correct these errors by supplying an 
explanation appropriate to the specific kind of error made 
right after the wrong answer. Such explanations and side- 
tracking hence need to be planned out in the same way as 
. the main questions. . ... . , 



(iii) Analysing the main questions as yell as the e xplanations 
and sidetracking makes it possible to present the 
questionnaire in an advanced stage of development, A sample 
Vi" worked out for the bionominal law is available for those 
questionnaire writers who may be interested, 

(iv) Writing the questionnaires in the final form in which it is 

given to the students occurs only at the last stage of develop- 
ment. The sole aim is to make the hypotheses and questions 
easy to understand. A second reading by people who are not 
familiar with the material is done to check that the state- 
ments are phrased clearly. 

(v) Two tables summarise conveniently the basic contents of the 
questionnaire. The first, which deals with the questions, has 
been set up in the following manner: 

Question Question Notions which Characteristic Uncharacteristic 

number content have to be expected expected 

understood in answers answers 

order to answer 



Contents of 
explanation 
connected to 
the answer 



Place where 
the answer is 
to be sent 



A second, "vertical" table can be set up in the same way. Its 
purpose is to classify the various concents rather th?n the questions. 

These two tables prove very useful in examining the anbwers. 



3. Some rules for design 

A goodly number of more or less contradictory requirements 
made it necessary to set up some fundamental rules of design which aim 
especially at controlling the outcome of the experiment better. 
Obviously these rules are not unalterable, and it would be wise to 
investigate their influence by testing questionnaires which adhere to 
different rules. .;.n ^ < . * . » * . « • 

(i) the total number of notions which occurs in any one questions 

; aire should be very email, if the student is to be able to 

• ^ check on whether he has assimilated all these notions, and if 
v it is desirable to find out efficiently from the results 
4 whether the student has assimilated them or whether his mis- 

■■ ■ " takes have been corrected. At this juncture, the exact number 

1 1 of such notions has not been dotemii el, tat it seems reason- 

* able to expect it to be considerably less than the total 
number of questions. * 

*•' (ii) Generally the student must have master^! several notions to 
answer any one question correctly. To trt vhethar a notion 
has been learneo (that is, can be usd c erectly), the same 
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notion must therefore appear in several answers, which can 
then he examined in a single hatch tied together by the notion 
being tested. For the moment, this is not an easy rule to 
apply, but it should make it possible to make a breakdown of 
the answers notion by notion. At any rate, it assumes that 
the analysis mentioned in III-1-B2 (ii) has been accomplished 
accurately. 

(iii) At most the questions should have bearing on types of reason- 
ing. Thus, it is necessary to avoid: 

- questions which essentially require the student to do a 
lot of figuring 

- questions whose answers can be memorised. 

These two points are studied by special questionnaires which 
raise specific problems. Nothing is gained by mixing them in 
with the others. This has soma repercussions on the choice 
of subjects for the questionnaires. In particular, in 
questionnaires dealing with problem solving exercises or with 
theorem demonstrations, it seems rather difficult to avoid 
the two kinds of questions mentioned above. It is therefore 
better, at least for the present, to avoid this type of 
questionnaire. 

(iv) Explanations connected with wrong answers serve to draw the 
student's attention to the mistake in reasoning which he has 
made. Hence, they should not contain the full answer to the 
question, but only an aid to correct the line of reasoning. 

The logical conclusion of this should be an "Answer once more" 
which allows the student to set his thinking on the right 
track and allows the author of the questionnaire to check 
whether the scheme for diagnosing errors is adequate. 

The sidetracking questions are put with the errors which 
have bearing on the student's understanding of one of the 
key concepts of the problem. 

(v) The policy was adopted of making no comments whenever the 
answer is correct. It is hence assumed that the student has 
understood the line of reasoning required to jive the answer. 
Studying all the students answers should make it possible to 
verify this hypothesis. It should be noted that this practice 
probably makes the teaching process less efficient. The 
student can indeed come up with the correct answer for the 
wrong reason. It would be possible to supply a commentary 
taking the form of "Right answer. Indeed ...% in which the 
right line of reasoning is explained. In this case, each 
student can compare his own thinking with the correct approach, 
and adjust his own thought processes if necessary. On the 
other hand, adding information in the form of explanatory 
commentary for accurate answers would obscure one of the 
. purposes of the experiment, that of seeing hov the student 
has learned and uses what he has come across in lectures and 
discussion sections. Thus, it is necessary to keep to a 
minimum the practice of supplying extra information (for this 
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type of experiment) and to strive to control its impact on 
what the student already knows (what he has gotten from the 
course, etc...). 

(v±) Repetitions in the way the questionnaires were stated were 

intentionally limited as much as possible. This has a number 
of drawbacks, especially on account of the form in which the 
questionnaire is shown to the student. It is worth noting 
that the reverse choice is a delicate business (useless and 
impossible lengthening of the form, or else limiting it to 
only those items required by the question - in which case, 
any attempt on the student's part to organise and sort out 
is excluded). This point ought to be studied with care. 

(vii) As few notations as possible were introduced, even if that 

means that the problem loses some of its scope. This was made 
indispensibla by the difficulties in machine writing and the 
dangers of improper confusions between notation used in a 
statement and a diffeient notation system which the student 
may have picked up elsewhere (for example, in the coursework). 
In any esse, the problem of having different codes has not 
been solved adequately. 

(viii) There U 3 still a touchy problem in how to process answers 

which were not anticipated* Experience shows that it is not 
enough to provide an explanation or a side track after such 
an answer, for that cannot remedy the kinds of mistakes which 
have not been identified. The only possible solution is an 
empirical one: making this category smaller by studying the 

answers supplied and by processing appropriately those answers 
which can already be diagnosed. This assumes close co-operation 
between the staff writing the questionnaires and that which 
examines the outcome - an understanding that those writing 
the questionnaires will foHow the work of analysing the 
results - a need to agree ->n the precise goals which the 
questionnaire is to reach before it is actually prepared. 



2. USING N1N1TRAN AND SYMPLK LANGUAGES. AND DISCUSSION CLASS Jj[0_RK 
IN MATHEMATI CS 



A. Ml nit ran and how it is used 

The presence of numerical applications in the questions which 
the system asked students during computer questioning sessions led us 
to conceive of a programming language which would be simple enough to 
be used by students who are not computer specialists and which would 
have a compiler appropriate for the small size of the memory store of 
our machine. 

At any point, the student can call upon the calculating apparatus 
by typing on his terminal the characters *CAL followed by his programme 
at the beginning of the line. Analysing and carrying out the operations 
are then done in a conversational pattern. One or several arithmetic 
expression^ can be evaluated, if need be with different numerical data. 
When the student receives the results of his calculations, he goes 
back to the examination session at the point he left it. 









Tho Mlnitran programme is written og^a single line of the control 
panel (120 characters;* This short programme option has the advantage 
of leading to swift compilations and operations. As a result, the time 
the efcudent needs to antwer is acceptable for both the control panel 
asking for the computation and for the other control console being 
used during the questioning session. 

The instruction "repeat 11 makes it possible to have some computation 
algorithms which includes a repetition with a definite number of loops 
(Fortran's DO type). 

The Minitran language was hitched up with the questioning system 
and has been functioning since December 1967. It has been noticed that, 
in actual practice, few instructional programmes use this possibility 
of milking computations. 

On the other hand, the ease with which this language is used made 
it possible for entering undergraduates (in the KP and PC groups:) 
during the 1967-68 school year to make practical use of this computation 
facility, aside from any examination purposes. 

Let us give an example of how this language is used. 

"Calculate the potential electricity produced by a basic charge 
e at a distance of 1 angstrdm". 

- Normal wav to write the problem ; V « 0 /4j[£ o ^ 

g 0 = 1/36 nlO" 9 KKSA 
R = 10 -10 METER 

e = 1 FARADAY/N o = 

96490/6.023 10 25 

- Programme's wav : 

*CAL | = 1/(36* n*lB9)' R» IB - if 

e = 9649*/(6.* 23*1B23)' 

V = e/(4*7I * 1 0 *R)' WRITE (V)’ 

In other words: 

"CAL V = 9649 $/6. 4 23£23/(IB - l4/9B9)' WRITE (V)' 



In actual use, Minitran proves not to be quite appropriate for 
introducing students to programming and methods of numerical computation. 
It has the following shortcomings: 

- the source programme is not sufficiently long 
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- there is no opportunity for partial correction of programmes 

. - there are no tags, hence no conditional or unconditional jumps 

- there is no panel. 

As a result, one is led to devise another version of Minitran 
and to add to it a control language, in order to make up for the 
weaknesses mentioned above. 



B. SviQDle and how it is us_ed 

A new conversational system called SYMPLE (System Multi-Pupitre 
a Langage fivolue) has been developed# Here is an example of how this 
language is applied* 



* Pro gramm e for solylng a second_degr.ee 



U = User 

U. /PRO 
S. READY 
U. 1 
U. 2 

U. 

u. 5 

U. 6 
U. 7 

U. 



S «= System 



&i READ (A,B,C) 

D = B*B ~ 4*A*C 
IP(D)5,6,7 

TXT (NO REAL ROOT) f VRSl 
TXT (DOUBLE ROOT) 

XI = (-B+/(D) )/2A* X2 = -B/A - XI 
WRITB(X1,X2) 



U • /EXC 

« 

3. 7 XI = (-B+(D))/2A'X2 = -B/A.- XI 

3. ERROR 6 COMPILATION CANCELLED 

• 

U. 7 XI = (-B+/(D) )/2A' X2 = -B/A - XI 

3. CORRECTION MADE 

• 

U. /EXC 

3. TYPE DATA PLBASB 
U. 2,3,2 
3. NO REAL ROOT 

« 

3. TYPE ^ 3 DATA PLEASE 



o 
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RLC 




X2 - -1 



U. 2,4,2 
S. DOUBLE ROOT 
S. XI = -1 
S. END 

• 

U. /INS 9,7 
S. READY 

U. 9 X = -B/2A' VRITB (X)' ART 

S. CORRECTION MADE 

* 

U, /LST 
S« 1 
S. 2 

S. 

S, 5 
S. 6 
S. 9 
S. 7 
S. 

• 

U. /EXC 

S. TYPE ^3 DATA PLEASE 
U. 2,4,2 
S. DOUBLB ROOT 

s. x = -i 

S. END 

• 

U. /EXC > • 

3. TYPE ^ 3 DATA PLEASE 
U. 1,4,2 

S. XI = -5857868 X2 = -3.414213 
3. END 

• t ■ 

U. 8 WRITB (XI, X2) 1 VRS 1 

• • • c - • . ■ 

U. /AST 

3. WRONG MACHINE OPERATION 
U. /EAV8 



READ (A,B,C) 

D = B*B-4*A*C 
IF(D)5,6,7 

TXT (NO REAL ROOT) 'VRS1 
TXT (DOUBLB ROOT) 

X = -B/2/A' WRITB(X) 'ART 

XI = (-B+»<D))/2/A'X2 « -B/A 
VRITB (XI, X2) 
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s. 

XJ. 

3. 

s. 



s. 



s. 



s* 

S • 

s. 

a. 



CORRECTION MADE 
/L ST 

1 READ (A,B,C) 

2 D = B*B - 4*A*C 
HIP(D)5,6,7 

5 TXT {NO REAL ROOT) 'VRS1 

6 TXT (DOUBLE ROOT) 

9 X = B/2/A' WRITE (X)» ART 

7 XU (-B +/(D))/2/A'X2 = - B/A - XI) 

8 WRITE (XI, X2V VRS1 



This programme vill be handled by the computer starting with the 
first line of instructions. The system thus starts by asking for the 
3 data which it will assign to A, B and 0 respectively. Then the value 
of D is calculated. Next, it considers 3 possible cases: 



- If D is negative, it goes to line labelled 4: print out a text, 
then go towards 9, that is, ask for 3 other data. 

- If B equals zero, move on to line 7: print out a text, then 
continue in order by line 

- If D is positive, move to line 5 : compute the 2 roots, then 
print them out. 

In the latter two cases, the command to stop is finally reached. 



C, Mathematics discussion section 

The Symple system is used for work in mathematics for small 
discussion classes, and for introducing programming. These sessions, 
addressed espeoially to beginning university students, took place two 
full days per week, and thereby we were able to accommodate several 
hundred students. 

The Minitran 2 conversational language is sufficiently simple for 
a student to assimilate it in a few hours. The first sessions were 
devoted to making up small programmes for ordinary computations, dealing 
basically with numerical applications but also requiring some logical 
decisions at the operations level. (For example: second degree equation, 
prime numbers The computer's resources were quickly put to use 

in order to work on less hackneyed programmes. 

Each session began with a brief report on the techniques of standard 
numerical computation. The interesting part is the comparison made with 
methods specially designed for automatic computation. Discussion of the 
degree of accuracy of the results, for example, can be the stepping 
stone to a detailed study of algorithms, and demonstrates the basic 
difference which exists between manual and automatic techniques (Example 
calculating a determinant, reversing a matrix, roots of algebraic or 
transcendent equations). 









In addition to purely numerical applications, logical manipulations 
were carried out, which led once again to choosing between algorithms# 
The problem of sorting a file supplied a particularly clear example, 
and gave rise to a wide variety of procedures depending on the type and 
size of the file involved. 

These sessions enabled the students to get a better idea of what 
uses a computer can be put to, and especially of the difficulties 
involved in formulating a problem accurately# The inflexibility of a 
computer makes it necessary to analyse a problem more thoroughly than 
usual. It is also worth mentioning that we organised similar sessions 
on a lower level when we invited in groups of high school seniors. Many 
of them got very excited (enough to come in on Sunday) and designed 
programmes with astonishing cleverness and spirit# 



3# BIQIOQY 



A. The group of biologists 

After a meeting at which biology professors were informed of the 
possibilities presented by the Student Computer Use (Ordinateur pour 
Etudiants - OPE), a group of six assistant professors of animal biology, 
teaching at the undergraduate level, got together to try an experiment. 
They chose a subject, for example, "Meiosis" in the animal kingdom. An 
initial questionnaire, which was short and sweet, was hammered out for 
the purpose of completing the handful of introductory technical exercises 
which the OPB gave to all student newcomers# 

A second questionnaire, far more complete, was made up afterwards# 

These questionnaires were submitted to various groups of under- 
graduate majors, and were also shown to a large part of the biology 
teaching staff to get their reaction. 

The six biologists had had no previous experience in the fields 
of computer-assisted instruction, computer techniques or data processing. 

It is rather different to calculate exactly how long it took to put 
together the questionnaires (getting the wording of the questions, the 
various answers that could be expected, and the corresponding explanatory 
comments) but certainly several hundred hours were involved. 

Next, it was necessary for the biologists to play at least some 
part in actually making the questionnaires (punching the cards, recording 
on the diskpack memory, etc...). They were readily trained in computer 
practices with the active co-operation of the entire OPB staff. 

Working out and physically preparing the questionnaires was done 
in work shifts which were added on to normal teaching and research 
schedules# 



B. The subject matter 

The very character of biology, its development, and its teaching 
needs raise speoial problems, some of Which have not yet been adequately 
solved# 
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In biology there are very few assertions which are true universally. 
Each so-called "general" rule usually carried with it restrictive clauses 
varying from one group or another, sometimes from one species to another 
or even from one individual to another. Peculiarities, anomalies, patho- 
logical cases tend to reduce even more the scope of a general law and 
make it all the more difficult to write questions and explanations which 
must be terse if they are to be used in the computer. 

It is likewise necessary to plan for *ome student answers which 
deal with such and such a specific case, not the most typical case, 

A single concept, fact or object in biology can be described in 
two or several ways [for examples synapsis and conjugation of chromo- 
somes). These various names coexist and in fact are used, so that 
students who have taken courses from a number of different professors 
have had to learn noticeably different vocabularies for the same subject. 

A single word can have various meanings, especially both a "broad 
meaning" and an "exact meaning". For example: Egg, Sometimes one term 
refers to objects or facts which are entirely different, depending on 
what group of animals or plants is being studied. (For example: tetrad). 
Furthermore, two names which one biologist considers to be equivalent 
are thought of by another biologist to be distinct, even if only 
slightly. Finally, a term may have undergone what linguists call a 
"drift in meaning" since it vao first coined or used, as a result of 
the development of knowledge and ideas. 

In virtually all branches of biology, drawings, graphs or photo- 
graphic materials mu3t be used. Without them, a student may be called 
upon to exercise his imagination and his ability to do abstract and 
conceptual thinking. But most of the time, doing without them results 
in description loaded down with tedious circumlocutions# Biology 
teachers claim, quite rightly, that in biology a good sketch is gener- 
ally preferable to a lonr explanation. 

It was possible to handle meiosis in the questionnaires without using 
illustrations, although analysing the processes of crossing-over, for 
example, which was considered briefly, would have raised dire problems. 



C. The main features of the questionnaires * 

The questionnaires developed are designed to allow the under- 
graduates to engage in "self-examination" after they took courses at 
the Facult^ des Sciences (lecture courses, discussion sections, class- 
room practice). The OPE session is optional. No marks or evaluations 
of any sort is given students by their usual teachers or by other 
parties. 

The first questionnaire, labelled by the symbol K_18,( f Introduction 
to Meiosis") has a simple form - it is made up of 12 questions in 
normcl sequence, without any jumping to side tracks. The purpose is 
to emphasise some general concepts, without going into detail on the 
mechanisms involved. It should enable the student to becomo aware of 
whether he has understood the meaning in biology of meiosis, whether 
he can locate it in a living organism, whether he knows the starting 
and finishing points, and so on. It also makes it possible to go into 
more detail on certain terms and notions. X 18 ends by announcing a 
more detailed questionnaire* 
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The second questionnaire, carrying the number K 68 ("flew programme 
on Meiosis"), is in fact made up of two sub-questionnaires - K 68 proper 
and K 19 - which must be automatically hooked together. It comprises a 
total of 31 questions, 21 of which are in normal sequence and 4 jumping 
on to sidetracks. 

It is concerned with exactly how meiosis occurs* "Reduction 
division" is dealt with, naturally, in a large number of questions. 

The essential notions - "pairing", "homologous chrono somes " , "chromatic 
reduction", "chiasma" , 11 crossing- over " , 1, autosome© ,, # "hetero chromosomes" 
etc. - emerge during the process, either from the wording of the 
questions or from the explanations* 

It ends with the notion of returning to the diploid state as a 
result of fecundation, and with the importance of "stirring up the 
gene pool" . 

A final survey should make it possible to know, in part, the 
students* opinior of the questionnaire and to receive their suggestions 
for future programmes. 

The first questionnaire, K 18 . being simple in design and con- 
struction, is acceptable for an "introductory 1 * session. It is no longer 
appropriate for other questionnaires, which are more complete and have 
higher ambitions. 

The second questionnaire, K 68 * with its two parts, is too long* 

To completely debug such a programme takes a long time, and involves 
drudgery, since each modification often necessitates other changes, 
and so on. 

K 68 should have skipped over the problems which were simple or 
easy to understand, merely referring them off to various sidetracks, 
and should have concentrated basically on a thorough study of a limited 
number of questions. 

Generally speaking, the questionnaires were designed to review all 
the important notions, whether they wore simple or complex. This plan 
has had its day* 

There are too many questions such as "Vhat do you call *.*?", 
despite an attempt to diversify nomenclature. Although questions of 
this type need not be excluded, especially when they concern difficult 
or new terms, it is usually best to have the student himself explain 
how facts work or occur. Nevertheless, the fact that an answer must 
not be longer than 58 characters imposes a limit in this direction* 

Thie problem will have to be studied once again. 

On the whole the questionnaires perhaps do not utilis-v fully all 
the present possibilities of the machine. In this- respect , it should 
be mentioned that in some future programme, the student may be called 
upon to make computations, and it will be easy to use the computer for 
this purpose. 

The multiple-choice question format was not used, vi\h just two 
exceptions. 



In nearly all cases, the student himself has to write in the key 
word or words, and therefore has to find them first in his memory or 
as the result of a chain of reasoning. 

The attempt to think and to actually build words, which to a 
certain extent is comparable to true re-creation , seems to be of 
fundamental pedagogical importance, even if the student is not entirely 
aware of it at the time. 

Each student must have a minimal vocabulary, without which he 
cannot study biology. For example, in K 18 , question 0£ (naming the 
cells in which meiosis can be observed) normally calls for the answer: 
"germinal" or "gerrainative cells' 1 . The answers, sexual cells", 
"reproductive cells" or "gametes" are unacceptable (they are expected 
in the answer-block), A certain amount of strictness is required. For 
question 15. in K 18 (naming the male cell at the end of meiosis), the 
answer "sperm" instead of "spermatozoid" is refused and an explanatory 
comment is given. 

The questionnaires are limited to the most important and general 
concepts and facts and to those which prove toughest for the students 
to understand. 

Experience in teaching often shows that thorough knowledge of a 
complex nomenclature is not necessarily connected to a sound under- 
standing of the mechanisms involved. The computer should not be used 
primarily for the purpose of aiding students to memorise long lists 
of words (even if that cannot be totally avoided). In particular, in 
the two parts of K 68 . some ten questions deal with prophase I . The 
usua’. nomenclature for the successive stages of this phase are neither 
mentioned in the questions and explanations, nor called for from the 
student (with the single exception of diacinesis). The characteristics 
and basic facts of this prophase I and what distinguishes it from a 
homeotypical prophase are emphasised. 

The questions vary in format and character. Either the name of 
some fact or object, or a terse description of a phenomenon is asked 
for. A few problems requiring rigorous thinking are raised, and some- 
times what may be called "traps" are set. 

The questionnaires are adjusted - albeit inadequately, it is true - 
to the spread of levels of knowledge and understanding which can be 
expected among students. 

The commentary which follows a mistaken answer sometimes asks the 
question again in a slightly different form, even adding new explana- 
tions (cf., gametogeneeis) • 



D, Results and student response 

In all, more than one hundred undergraduates participated in 
answering the first questionnaire ( K 18 7. About a third of them were 
able to return to do the second one ( K 68 + K 19 ). Various adverse 
circumstances - in particular, the final examination period - prevented 
the others from coming back, although they wished to do so. 
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A session for each student lasts usually from 30 to 43 minutes. 
Those few students who remained for a full hour began to feel tired. 

It is worth noting that part of the reason the students showed 
such keen interest in this experiment is that all technical and 
pedagogical novelties are appealing. 

Although the students felt constrained by the physical difficulties 
of handling the control panel, they rapidly came to feel at home. 

' The experiment showed that the student should feel free and 
unobserved while he ?s carrying out his "self-examination”, and the 
staff should intervene only when called upon. This is especially 
reasonable in that the computer gives the student the correct answer 
after two mistaken answers. 

At the end of a session, the computer is able to supply a statis- 
tical inventory containing the various types of answers which were 
given to each question, the average time the student took to answer, 
and a list of all answers which were not anticipated. 



In the present state of development, the most interesting data 
came from personal conversations with the students after the various 
sessions. Useful albeit fragmentary information has already been 
obtained in this way# 

The students were duly informed that this work was experimental 
in nature, that they were considered to be fellow researchers rather 
than pupils, and that their reactions were going to make it possible 
to improve the questionnaires, develop others with different patterns, 
and help the teaching staff to define its most effective role. 

The students themselves expressed the sense that they were 
’’relaxed 1 ' around the machine. 

As a rough guess it would seem that a student who is not sure of 
an answer is more willing to say so than he would be to a professor. 

Students who feel free to express themselves, and are required 
to seek their answers actively, seem likely to learn more readily. 

The details of this process have not yet been demonstrated and under- 
stood. 

Furthermore, it can be assumed that if the student forms the habit, 
by this method, of answering more readily, he will apply it in other 
kinds of examinations and will have increased self-confidence. 

• : „ % X. „ „ . ( ^ _ 

Vrom the very outset, students are overwhelmed by the need to 
give accurate answers. In traditional examinations, they can beat 
around the bush, little by little coming closer to the solution, and 
finally managing to make themselves understood with the aid of a "you 
see what I mean” or two. That is not the case here. 

The undergraduate biology majors find the need to te rigorous so 
new and baffling thac at first they do not really know whether they 
should be delighted or dismayed by it. After a while, they generally 
end up favourably inclined on this point. 
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There are two contrary perils which must be steered away from: 

- first, personification of the computer, which has the tendency 
of ascribing human behaviour and responses to it. 

- second, mechanical coldness under the pretext of being "neutral", 
which in the long run would be disagreeable. 

The students seemed to be satisfied with the balance finally 
reached in the questionnaires. 

An OPE SQL jior. is useless and not to be recommended for students 
who have not studied "Meiosis". Students who studied this topic 
specially in oruer to prepare for a session did not benefit as much 
as they expected to. It happens that the questionnaires were not 
designed for that level of knowledge, despite the attempt to give them 
a vide scope. One of the tasks in the future will be to add more 
variety to their structure to make them appropriate for a wider audience. 

The session is clearly beneficial, above all, for those who have 
studied Meiosis the same amount as they have other parts of a regular 
course, and who as a result are still in the dark or are unsure of 
certain points. In the last analysis, that describes the majority of 
the students who came to the OPE. 

The students are very interested in the experiment and are gener- 
ally convinced of the effectiveness and future outlook of the method. 

They all hope it will be expanded widely. 

Some students seemed troubled by the prospect of a mechanical, 
automated universe which looks like it would have to eliminate human 
thinking and initiative little by little. At any rate, such pre- 
occupations can not be overlooked. 

Some of the questions were not formulated in a satisfactory manner. 
As a result, the questionnaires were reworked whenever it seemed 
necessary. 

After an inaccurate answer, instead of giving out an all-purpose 
formula such as "Think and answer once again", the computer ought to 
ask the question again in a different form, if need be with new 
information. 

This observation is probably one of the most pertinent and con- 
structive ones made. From the outc t. this process was used occasion- 
ally, but not often enough. Since then, the questionnaires have been 
changed appreciably in this direction, at least for some questions. In 
the future, this problem will attract the attention it deserves. 

Some questions appear to be just about useless, when their ansve 
are too easy or too obvious. Although this observation is justified, 
it was not possible to change the questionnaires accordingly, since 
the whole concept of the experiment was involved. Future programmes 
will be based on entirely different patterns. 

Students often wish to engage in a dialogue with t he computer. 

More particularly, they would like to be able to ask the machine 
questions, request definitions or even the complete explanation for a 
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